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ABSTRACT 

A study assessed the selection techniques used to 
secure focus group respondents at one site as part of a larger study 
to determine what viewers learned from the 1992 Presidential Debates, 
A two-stage random sampling technique was developed to select focus 
group respondents from the 224,041 registered voters in Syracuse, New 
York, The random selection procedure was used to select 88,8% of the 
prospective participants. The remaining prospects were selected from 
a stratified list to compensate for under-represented members drawn 
from the random sample, A total of 38 respondents (6.3yi) agreed to be 
respondents as a result of telephone calls made to prospective 
respondents. Last-minute declines by individuals and their 
replacement by participants who were conveniently available skewed 
the representativeness of the 29 professionals, students, 
self --employed people, homemakers, and retirees who actually 
participated in the focus groups from the demographic characteristics 
of the community on the variables of gender and party affiliation. 
When applied to the study of non-homogeneous populations, as voters 
are, focus group research must approximate the representativeness 
standards normally associated with survey methodology. While it is 
premature to comment on the complete content analysis of the 
presidential debate focus groups, a cautionary note should be sounded 
to refrain from over-generalizing the results. Focus group 
methodology, appropriately adapted, can yield both qualitative 
understanding of political phenomena as well as quantitative 
general izabi 1 i ty, (Six tables of data and nine footnotes are 
included, (Contains 12 references.) (RS) 
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A DESCRIPTIVE ANALYSIS OF FOCUS GROUP RESPONDENTS 
IN THE 1992 PRESIDENTIAL DEBATES 

Presidential Debates have been a continuous fixture in 
American presidential elections since 1976. While some variety 
in format had occurred in presidential primary debates, the 
Presidential Debates had used a panel of journalists as ques- 
tioners since their inception in 1960 (Hellwig, et al 1992; 
Lanoue & Schrott 1991). The 1992 Presidential Debates introduced 
two ^eatures which had not previously been used. The first 
feature was the use of multiple formats to conduct the debates. 
The second feature was the commissioning of focus groups by the 
Commission for Presidential Debates to evaluate how voters used 
the debates to inform their decisions. The combination of these 
two features created an opportunity to assess the effect of the 
several form.ats on voters. 

We have limited our analysis in this paper to an assessment 
of the selection techniques used to secure focus group respond- 
ents. The transcription of focus group tapes is underway and 
should be completed this summer. Only preliminary results drawn 
from the focus group facilitators are available at this time. In 
order to locate the relevance of this analysis to the larger 
project, we will briefly describe the purpose and procedures 
employed in the study. We will then describe the selection 
techniques used at one site and how ( in) adequately it represented 
the larger community from which it's sample was drawn. Finally, 
we will report the selection technique used at the several other 
sites and evaluate the limitations of non-representative sam- 
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pling. Implications for the generalizability of focus group 
research in political campaigns will conclude our discussion. 

Presidential Debate Focus Group Study 
The larger research project was funded through a grant from 
the Commission for Presidential Debates. Prof. Diana Carlin, 
University of Kansas, was the grant recipient and principal 
researcher. The purpose of the research was "to determine what 
viewers learn from debates and what reactions participants had to 
the formats used in the debates.'* (Carlin 11/2/92) 

Fifteen sites were initially selected to conduct focus 
groups on the debates.' A total of 497 respondents participated 
in forty-eight focus groups. Most of the sites conducted three 
focus groups drav;n from the following five targets: The debate 
about the debates (late September);'' 1st Presidential Debate 
(Oct. 11); Vice-Presidential Debate (Oct. 13); 2nd Presidential 
Debase (Oct. lb); and the 3rd Presidential Debate (Oct. 19). 
Three formats for focus groups were used with at least five sites 
assigned to use each format: Format #1 used the same group 
members for all three debates (Panel Format);- Format #2 used 
different individuals in each session; and Format #3 ran two 
focus groups simultaneously segregated by gender. 

Instructions provided to Focus Group Facilitators included 
directions for recruitment of participants. Facilitators were 
provided with a standardized script and requested to select 
individuals who represent a cross-section of the community. It 
was recommended that facilitators select individuals from voter 
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registration lists, if available, or through the use of randomly 
generated phone numbers from the exchanges used in the community. 
Facilitators were also asked to balance the number of men and 
women participating in the focus groups as well as represent the 
party affiliation of local demographics. 

Individuals who participated in the focus groups reported to 
a designated location to watch the debate together.'^ Where 
possible, facilitators selected C-SPAN as the preferred channel 
for viewing the debate to minimize the influence of commentary. 
In any event, the volume was not turned on until the beginning of 
the debate and was turned off at the conclusion of the debate. 
At the conclusion of the ninety-minute debate, participants were 
given a short break and light refreshments before convening the 
focus group. 

When participants first arrived at the site, they were asked 
to read and sign a consent statement. Participants also com- 
pleted a questionnaire in several stages. The first stage was 
completed before the debate and consisted of several demographic 
items, their perceived level of exposure to campaign coverage 
during the past 6-8 months, and the top chree sources of campaign 
information. Participants were asked whether they had a can- 
didate preference (but not who it was) . The second stage occur- 
red immediately following the conclusion of the debate. Here 
participants were asked whether their candidate preference had 
changed as a result of watching the debate. The third stage was 
conducted after the conclusion of the focus group discussion. 
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Participants were asked whether their candidate preference had 
been altered as a result of the focus group discussion. 

The focus group discussions were (audio) tape-recorded. A 
standardized schedule of questions was provided to each facilita- 
tor to direct the discussion. Topics for discussion included 
quef5tions about other sources of information participants used to 
gain knowledge of the candidates and issues, the influence of 
press coverage, their assessment of the debate formats, and 
recommendations for future presidential debates. 

At the conclusion of the focus group discussion and comple- 
tion of the questionnaire, participants were thanked and dis- 
missed. Each participant received a nominal honorarium as a 
token of thanks and to help defray any costs involved with their 
participation. Summaries of the discussion and questionnaires as 
well as the audio-tapes were sent to the principal investigator. 

Selection P r ocedure and Results from Syracuse 
The authors were recruited by the principal investigator to 
serve as facilitators for the study in late August. The city of 
Syracuse is located in Onondaga county in central New York. 
Onondaga county has a population of approximately 460,000 resi- 
dents and is the largest county in a three-county SMSA. It was 
decided that registered voters in Onondaga county would consti- 
tute the sampling frame from which focus group participants would 
be selected. The Onondaga County Election Board (OCEB) made 
voter registration lists available for use in the study. 

The voter registration lists had recently been updated in 
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anticipation of the November election.^ While voter registra- 
tion continued during the selection of participants, the regis- 
tration lists were not revised until after the election. Conse- 
quently, the sampling frame remained constant during the course 
of the selection process. The voter registration lists were 
printed on 4,228 pages with 53 names per page enrolling 224,041 
voters. A two-stage random sampling technique was deve.oped in 
which first a page number (1-4228) and then a page location 
number (1-53) was selected. Krueger (1988) recommends the use of 
random or systematic sampling procedure for the selection of 
respondents . 

Among other information, the registration lists included a 
voter's name, gender, party affiliation, address, and phone 
number. As prospective participants were drawn from the regis- 
tration lists, a record was made of the above information. Over 
six hundred prospective participants were eventually selected 
using this procedure.'' 

As the principal researcher had instri ted facilitators to 
represent a cross-section of the community, we relied upon the 
random selection procedure to select almost ninety percent 
(88.8%) of our prospective participants. The remaining prospects 
were selected from a stratified list to compensate for under- 
represented members drawn from the random sample.' 

The demographic characteristics of the sampling frame on the 
variables of gender and party affiliation are compared with the 
demographics of the prospective participants in Tables 1 and 2. 
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The sample characteristics roughly approximated the sampling 
frame from which they were drawn. 

Table 1 
DEMOGRAPHICS BY GENDER 



Male Female Total 

Sampling Frame 45.6% 54.4% 100% 

( 99, 931) (119, 059) (218, 990) 

Sample 42.3% 57.7% 100% 

(256) (349) (505) 



Note,: There were 5,101 voters not identified by gender. 



Table 2 

DEMOGRAPHICS BY PARTY AFFILIATION 





Dem 


Rep 


NE 


Con 


Lib 


RTL 


0th 


Total 


Sampling 


















Frame 


30.2:; 


42 .4'^ 


2A.2%. 


1.4i 


0.8% 


0.4% 


0.6% 


100% 




67, 658 


94, 991 


54, 172 


3,196 


1700 


922 


1402 


224, 041 


Sample 




AO. 2% 


2 6.8% 


1.5% 


0.5% 


0.5 


0.2% 


100% 




(184) 


(24 3) 


(162) 


(9) 


(3) 


(3) 


(1) 


(605) 



Note,: Percentage for Sample exceeds 100% due to rounding. 



Note?: Dem = Democrat; Rep = Republican; NE = No Entry (Indep.) 

Con = Conservative; Lib = Liberal; RTL = Right to Life; 
and 0th = Other 



Actual participants were recruited from the sample in the 
order they were selected using a script provided by the principal 
researcher. We made no attempt to contact individuals who had no 
listed phone number.^ Other individuals were not contacted when 
their selection would have over-represented their gender or party 
af f i liation. 
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As prospects were called, a record was kept to note the 
outcome of the phone call. The record included notation of 
whether contact with the target was achieved. In instances where 
no contact was made, notation included whether there was no 
answer, a busy signal, an answering machine, etc. 

When we were successful in contacting a person other than 
the designated target, we attempted to establish an alternate 
time to call back. In those instances where a third party 
reported the target would not (or could not) participate in the 
focus group, we recorded the reason offered. 

When contact was made with the target prospect, we recorded 
whether the person was willing to participate. If the target 
declined to participate and offered a reason, we recorded the 
reason. We did not ask for a reason unless the target volun- 
teered one . 

Prospects who agreed to participate in the focus group were 
asked to confirm a mailing address so that additional information 
could be sent. A follow-up phone call within a day of the 
scheduled focus group was made as a reminder of the time and 
location. Table 3 reports the breakdown of attempted contact by 
category . 
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TaMe 3 

CONTACT ATTEMPTS BY CATEGORY 

No Attempt Unsuccessful Decline Accept 

Subjects 16.7% 36.2% 40.8% 6.3% 

(101) (219) (247) (38) 



We were interested in the reasons individuals declined to 
participate. As our notations recorded these reasons, we used a 
content analysis scheme to code the reasons offered. A coding 
sheet was developed to represent the reasons individuals offered. 
The coding scheme allowed both reasons offered by a third party 
as well as by the target. Three coders, not affiliated with the 
project were trained to analyze the record sheets. An acceptable 
inter-coder reliability was obtained using Scott's pi (81.5%). 

About one-third (35.1%) of the 247 targets who were success- 
fully contacted and declined to participate offered a reason. 
Table 4 reports the reasons of targets who declined to parti- 
cipate . 

Table 4 

SUBJECTS WHO DECLINED AND OFFERED A REASON 

Pol Age Trans Sched Time Fear Other Total 

Reason 2.3^. 7.0?^ 3.5?. 44.2% 7.0% 2.3% 33.7% 100% 
(2) (6) (3) (38) (6) (2) (29) (86) 

Note^: Pol = Political; Trans = Transportation; Sched = Schedule 

While 38 individuals initially decided to participate in the 
study, only 29 (75.3^o> actually materialized for the focus group. 
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In most instances we were unable to determine why individuals 
backed out of the focus group. Of those who backed out for which 
a reason could be determined, one backed out because of a sick 
parent. The spouse of another subject who backed out explained 
that his wife changed her mind at the last minute because "it 
really doesn't matter." Finally, a third individual backed out 
because, as a federal employee, his supervisor advised him that 
his participation could be in violation of the Hatch Act. 

Last minute declines by individuals and their replacement by 
participants who were conveniently available skewed the repre- 
sentativeness of the focus group participants from the demo- 
graphic characteristics of the community on the variables of 
gender and party affiliation. Tables 5 and 6 report the break- 
down by gender and party affiliation of the actual focus group 
participants . 

Table 5 

GENDER OF FOCUS GROUP PARTICIPANTS 





Male 


Female 


Total 


Sampling Frame 


45.6% 


54 . 4% 


100% 




( 99, 931) 


(119, 059) 


(218, 990) 


Sample 


42.3% 


57.7% 


100% 




(256) 


(349) 


(605) 


Participants 


55.2% 


44.8% 


100% 




(16) 


(13) 


(29) 
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Table 6 

PARTY AFFILIATION OF FOCUS GROUP PARTICIPANTS 





Dem 


Rep 


NE 


All Other 


Total 


Sampling 












Frame 


30.2% 


42.4% 


24.2% 


3.2% 


10 0% 




(67, 658) 


(94, 991) 


(54, 172) 


(7,220) 


(224, 041) 


Sample 


30. 4% 


40.2% 


26.8% 


2.6% 


100% 




(184) 


(243) 


(162) 


(16) 


(605) 


Participants 


37. 9% 


34 .5% 


24.1% 


3.41 


100% 




(11) 


(10) 


(7) 


(1) 


(29) 



Note;: All other equals the combination of Conservative, Right 
to Life, Liberal, and Other from Table 2. 

While the gender and party affiliation skews may seem 
nominal in the aggregate of the three focus groups conducted at 
this site, the composition of the individual focus groups are 
more distorted. Of course, the two known variables can be 
compared against a population norm. Other characteristics, for 
which population norms are less well established within the 
sampling frame, are also likely to be skewed. For instance, the 
occupations reported by the twenty-nine participants in the focus 
groups are broken down into the following categories: Profess- 
ional (14), Self-employed (3), Homemaker (3), Retired (3), 
Student (2), Service (2), and one each Blue Collar and Un- 
employed- 

In the final section of this paper we evaluate the generali- 
zability of focus group results. The selection methods for 
obtaining respondents influence the ability to generalize the 
presidential debate focus groups to the general population. 
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Limitations to Representativeness and Generalizabi!itv 

We have described in some detail the methods used for the 
selection of focus group members at one site for the recent 
presidential debates. Reports from other facilitators par- 
ticipating in the project indicate that random selection methods 
were not used uniformly. Some sites used random digit dialing to 
obtain the bulk of their participants while others relied on 
voter registration lists. Others abandoned random selection 
methods when preliminary yields were disappointing.^ 

One question which needs to be addressed centers on whether 
the mix of selection methods will ultimately yield results from 
which valid and reliable generalizations about the presidential 
debates can be obtained. Another issue focuses upon whether the 
assumptions upon which focus group methodology is based need to 
be reconsidered when applied to the study of non-homogeneous 
targets. We will address each of these in turn. 

The question of generalizabil ity depends upon two assump- 
tions. First, the sample must be adequately large in order to 
control for statistical artifacts. Second, the sample must be 
representative of the target population so that the normal 
distribution of elements will be observed. (Kish 1965) Our 
concern here is with the second assumption. As McClosky (1967) 
notes: "In general a sample must more perfectly reflect the 
characteristics of the universe being studied if the investigator 
wishes to describe that universe than if his main concern is to 
discover or test relationships among variables." (68) 
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Focus Group Responcientsl2 
Selection techniques which do not ensure the representation 
of the population in their sampling limit the generalizabil i ty of 
the study. Random selection techniques are not the only method 
for securing representativeness. However, when the phenomena 
being studied are the undifferentiated impressions formed by 
voters, it behooves the researcher to reflect the universe to 
which the impressions will be generalized. 

The question of generalizability becomes a central concern 
when the assumptions of focus group research are evaluated. 
Focus group research originated in market research in which 
specific market segments were the target of investigation. 
Consideration of sample size (Libresco 1983) and representative- 
ness (Merton et al 1990) are less valued than homogeneity and 
availability (Stewart and Shamdasani 1990) . Rosenstein (1976) 
noted the methodological weakness of focus groups is in the 
subjectivity employed in conducting the group as well as their 
lack of replicability . 

When applied to the study of non-homogeneous populations, as 
we contend voters are, focus group research must approximate the 
representativeness standards normally associated with survey 
methodology. while it is premature to comment about the yet to 
be completed content analysis of the presidential focus groups, 
we believe a cautionary note should be sounded to refrain from 
over-generalizing the results. 

Focus groups may hold promise for the analysis of political 
debates. Focus group methodology, appropriately adapted, can 



14 



Focus Group Respondentsl3 
yield both qualitative understanding of political phenomena as 
well as quantitative generalizability . However, to accomplish 
this, procedures suitable to survey research must be employed. 
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Endnotes 



The initial fifteen sites were subsequently expanded to 
seventeen locations to include the debates sites. The sites 
were: Sacramento, CA; Boston, MA; Atlanta, GA; Sioux Falls, 
SD; Spokane/Cheney, WA; Tempe, AZ; Durham, NH; Lawrence, KS; 
Hattiesburg, MS; Detroit, MI; Stephenvilie, TX; Cincinnati, 
OH; Kansas City, MO; Richmond, VA; Tampa, FL; St. Louis, MO; 
and Syracuse, NY. 

The original schedule of debates proposed by the Commission 
for Presidential Debates were for presidential debates on 
September 22, October 4, and October 15 with a vice- 
presidential debate on September 29. When the candidates 
could not agree on dates and formats, the principal re- 
searcher scheduled focus groups to record the public's 
perception of the debate about the debates. Shortly after 
these focus groups were scheduled, the candidates finally 
agreed on dates and formats for the debates. 



The term "panel*' is used as a standard designation for the 
repeated use of the same subjects. (Babbie 1992) 

For the "Debate on the debates" focus groups, participants 
did not have anything to watch on television prior to the 
focus group. 

The registration lists were current through September 22, 
1992. 

There were four participants who were selected by non-random 
techniques to substitute for participants who withdrew at 
short notice. 

The first 536 names were drawn from the random selection 
procedure. The remaining names (69) were selected from a 
random selection, but only frcm among women who were regis- 
tered as Democrats, Liberals, and No Entry (Independents) . 

The sample included individuals who had no phone number 
recorded on the voter registration list. We attempted to 
secure a phone number from the directory. In most cases 
we were unsuccessful. 

A request for information from facilitators on how they 
selected participants yielded responses from 8 (of 17) 
sites. Of those responding, only four used random selection 
methods as the primary means of securing participants. 
Other methods included "convenience" sampling and door-to- 
door solicitation. 
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